RasPi Beats
Rafael Gottlieb (rdg244) and Chimdi Anude (cca54).
ECE 5725 | May 14 2024
Demonstration Video
Introduction
RasPi Beats is an interactive drum beat maker. The embedded system uses an LED panel, a display, and a button matrix to make drum beats. The system allows you to interact with eight tracks simultaneously, each consisting of 16 beats. The sound loops through each beat, allowing consistent playback. The system can play up to 13 sounds, with each track being able to change between the different sounds when toggled.
Project Objective:
This project aimed to incorporate a Raspberry Pi into an interactable music creator. Learning an instrument is challenging; however, this project aimed to create an easy-to-use interface to simplify music-making. This project aimed to use a mix of hardware and software interaction with a Raspberry Pi.
Design and Testing
Hardware
The system comprises 4 main pieces of hardware: an LED Panel, the piTFT display, a button matrix, and a 3D-printed case. The case was designed using OnShape, a web-based CAD software. This case holds all components of the project. The top section has the name of the project and the holder for the Raspberry Pi and piTFT (placed on top of the Pi). There is a slot for the wires connecting to the LED panel and a slot for the USB connections to the Pi. The case was split in half to remove and fix components for debugging easily. The case was printed in the Rapid Prototyping Lab at Cornell and was made out of PLA.
As seen in Figure 1, the Adafruit LED panel is a 16 x 32 LED matrix, with each LED spaced 1 cm apart. The panel is connected to an outlet through the back of the panel to power the LEDs and connects to the Pi using a Hub75 driver. The GPIO pins connected to the Hub75 are as shown in Figure 3.
The piTFT display is connected to the Raspberry Pi using the pins in Table 1. From looking at the connections of the LED panel and the piTFT, pins 17, 22, 23, and 27 are shared between the two, and could cause issues. However, these pins are associated with the physical buttons on the piTFT, meaning that without interaction with these buttons, no changes occur. Because of this, the project was designed to not use the physical buttons on the piTFT such that the LED panel can use the shared GPIO pins.
The last component is the button matrix. The LED panel was split to associate 4 LED’s with one button. This meant the matrix consisted of a total of 128 buttons: 8 rows and 16 columns. As mentioned earlier, a row was associated with a track, and a column was associated with a beat. This meant that upon pressing a button, the track would play at that beat. In order to determine a button press, the buttons of each row were connected in series from a 3.3V power source and a 10kOhm pull-up resistor to a GPIO input pin. Additionally, the buttons of each column were connected in series from a 3.3V power source and a 10kOhm resistor to a GPIO output pin. The wiring diagram for a 3x3 button matrix is shown in Figure 4 using an Arduino Uno. Because the button matrix for RasPi Beats is an 8x16, there must be 24 GPIO pins, one for each column and row. An Arduino Uno only has a maximum of 22 GPIO pins without a multiplexer. Because of this, an Elegoo Mega 2560 microcontroller was used (the same as an Arduino Mega 2560). The buttons were connected with a thin wire provided by Professor Skovira. This made the connections easier to solder and minimally cover the LED panel, as seen in Figure 1(A). The team also plugged a speaker into the audio jack of the Raspberry Pi to play the music.
Software
The team used Python for the software because the team wanted to use the PyGame library for the PiTFT display and audio files. Our software has four main sections: loading the audio files, initializing the piTFT, LED panel, and button matrix, the thread callback function, and the piTFT display levels. Before writing any code, the team recorded one-second audio clips of 8 different drum sounds (Bass Drum, Crash Cymbal, Cymbal, Hi-Hat with stick, Hi Hat 1, Hi Hat 2, Side Snare 1, Side Snare 2, Snare, Snare Tom, Tom 1, Tom 2, Tom 3) and saved them as wav files. The team specifically used pygame.mixer to process these clips to make them Sound objects that the team could play on the speakers. The team initialized the first 8 instruments to the tracklist (which were the eight rows on the LED Panel) in a list called curr_channels. The full list of all the audio files as Sound Objects was named all_channels. The second section was initializing all the hardware components the team was using. For the piTFT, this meant creating variables for the buttons the team wanted to display on the piTFT. The team also imported a custom logo for our project as seen in Figure 5 below.
To initialize the 16x32 LED Panel, the team used a previous student’s project website that used the same panel[3]. The team filled the entire board to pink to match the color of the Raspberry Pi in its initial state before any buttons were pressed as seen in Figure 6.
The system used all the pins on the Raspberry Pi to wire the LED Panel, so another microcontroller was implemented to program the button matrix and have it send the information to the Raspberry Pi. The team used the Elegoo Mega 2560; this required the team to write the button matrix decoding in C. From there, the coordinates of the button that was pulled low were sent to the serial monitor. The Raspberry Pi would then watch the serial monitor, and when it gets a coordinate it will toggle the row and column of the button matrix between True or False depending on the previous state. In the Raspberry Pi’s matrix, it was just an 8x16 array of Boolean values initialized to False. The third section was the callback function thread. The columns for the LED Panel represented 16 beats. This callback function runs the LED panel and the audio playback. The function runs through each row in the current column (the column in this case is the current beat) and determines if the button was pressed. When a coordinate was sent from the Elegoo Mega 2560 due to a button press, the corresponding coordinate on the Raspberry Pi would just toggle. When the button was pressed, its corresponding LED area would turn blue. If it has been pressed, then the LEDs associated with the button will change from pink to blue, and the audio of that entire column if activated will play. As a visual representation of the columns being played one by one, that entire column is filled in as white except if a track in that column has been toggled as seen in Figure 7. The team specifically used those threads because the team believed that it would help with timing more efficiently than using an interrupt service routine. The team used a sleep function to time the thread to be whatever the current BPM was set to. How the team tested this is simply playing with the buttons and hearing if the correct sound would play through the speakers and if the team pink back to pink, no sound should play.
The final section of the software was the piTFT display. The display was broken up into three levels: the Home screen, the Main Editor, and the Change Audio. The home screen as seen in Figure 8 simply featured the logo created on Canva, a start button, and a quit button to go back to the terminal.
Level Two which is called Main Editor as seen in Figure 9 is where the user can make the audio go faster or slower by changing the BPM of the callback function thread. Pressing Back takes the user to the Home Screen. You could also change what track you want to edit by pressing the Up and Down buttons. Once you find the track you would like to edit, press the GO button to go to Level Three.
On Level Three as seen in Figure 10, you can change the audio of the selected track to any of the thirteen drum sounds made available. When you enter Level Three, the LED Panel pauses hence stopping the music so you can hear each drum sound before pressing Save and returning to the Main Editor. How the team tested it was individually going to each track changing the audio on Level Three and playing it in Level Two to see if it was playing the correct drum sound.
The code was programmed to run for a certain amount before it times out, ending the callback function thread and quitting the program altogether. Testing our code meant trying to use it the way the team intended. So, the team produced simple beats to see if the button matrix functionality was corresponding to the correct audio files and that even if the audio changes the right track is still being played. The team also increased the BPM to see if the program was playing the 16 beats faster with every increase.
Results
The demonstration of the project did not match the goals. The expectation was to have the display update in real time while the audio was playing simultaneously. This proved to cause issues with the audio playback. With simultaneous editing and playing, the audio would produce an error, as shown in Figure 5. This would cause a delay in the music playing and prevent the playback from running smoothly. During the demonstration of the project to the class, this continued to arise, preventing the beat maker from running at a consistent bpm.
This caused the team to test many different Python libraries. Because each sound had an individual wav file attached to it, playing the files simultaneously was a problem. One attempt was to use Pydub. This library has a function called overlay, which allows the audio to overlay on top of one another before playing the sound. This was beneficial to have perfectly synced audio, however, Pydub does not allow the audio to stop playing until the file is complete. Each audio file is 1 second long. To control the BPM of the beat maker, there was a need to either control the speed of the file or to stop the file from playing. Pydub has none of these functions, therefore it was not beneficial to our application.An additional option the team considered was to use the Scipy library to control the wav files. Scipy has a library called Wave that allows the program to control the wave file. This library allows the user to read or write the file and change aspects of it. However, the team needed to both read and write the file to change the frame rate of the file, and then play the file. However, the library only allowed the team to either read or write the file, preventing the team from using this method. Before the final method, the team tried using an additional library known as Pyaudio. This library gave the team the ability to decode the WAV file into an array of integers. It created a stream that would input a byte array to produce sound. To allow for simultaneous sound playback, each WAV file that would be played on a specific beat was decoded into an array of integers and then weighted equally into a new array of integers. This allowed each file to have an equal volume. This was then converted to a byte array and played through the stream. This is shown in Figure 12.
This past method led to issues with the use of the stream function, and had the same issue of PyAudio, with the files unable to speed up or stop the audio. Because of this, the team stuck with using Pygame for the audio. Though this method did not play the sounds simultaneously in the program, the for loop through each row was too quick for the human ear to discern. This method stored the wav files in a class called pygame.mixer.Sound, which then allowed the program to play when it needed, but also to stop the playback after the beat was finished. This meant that in theory, the team were able to sync with the bpm, as the team could stop the audio file when the beat was over. However, this still led to the delay issue discussed in Figure 5. Upon running the demo for the website, the piTFT display froze. This prevented the team from being able to interact with the display and change the button presses. However, the audio played at a consistent bpm, running smoothly for the first time. This led the team to believe that the problem with the audio running was due to updating the piTFT and the Pygame displays. To test this, instead of reading the serial monitor for buttons pressed, the team created an initial drum beat. In the main while loop, all the code related to the piTFT was commented out, leaving only the time indicating when the loop timed out. This fixed the problem, allowing the audio to play synchronously and at high speed. Below is the same beat at 120 BPM and 500 BPM.
RasPi Beats at 120 BPM
RasPi Beats at 500 BPM
With the way the project was developed, real-time editing and playback is not possible. To use the current setup, the code needs to be altered to have an initial while loop that sets up the information for the beat maker: what sound is on each track, what buttons are pressed, and what speed you want to play. However, when this is done, the program will need to move to the next loop that runs for a set amount of time, and then times out. This will allow the current code to run with smooth, consistent audio. A final program was written following these changes. The initial state of the system allows the user to interact with the buttons and the piTFT display, and then when it has finished, the play button is hit, beginning the playback. Currently, the system cannot stop unless the user turns off the Raspberry Pi, or the program times out. However, future edits would allow the user to toggle between playback and editing.
Final Results
Conclusion
In conclusion, the team was able to make an interactive drum machine using a Raspberry Pi as the main microcontroller. Despite RasPi Beats not working to its fullest potential, the team can confidently say that this project has solidified their understanding of Linux and embedded operating systems as a good culmination of everything that was learned this semester. The team acquired a lot of knowledge and resilience and is grateful for this opportunity to have more hands-on experience.
Future Work
As for future work on this project, the team would like the opportunity to focus on two objectives: fixing the timing issue and adding more instruments to the catalog. Focusing on the first objective, as mentioned earlier, the timing between the 16 beats was not even, so when trying to produce a beat, it would sound chaotic due to the uneven timing. We played around with different audio libraries, and the bug persisted. If we had more time to explore, we would try rewriting the piTFT display as, towards the end, we realized that it was contributing to the unnecessary delays. That way, when the beats per minute increase, it sounds like music, not a bunch of beats strung together. For our second objective, producing a sound with different instruments, including a piano and a guitar, would be cool to produce a band on the Raspberry Pi. We would have to explore different levels of the piTFT display to give a display for all instruments and how to store all those audio files.
Team
Project group picture
Chimdi Anude
cca54@cornell.edu
Worked on the software
Rafael Gottlieb
rdg244@cornell.edu
Worked on the hardware and the audio playback
Parts List
The total allowable budget was $100. With parts acquired from Professor Skovira and ourselves, the project was well below the maximum budget.
- Raspberry Pi - Provided in Lab
- piTFT Displa - Provided in Lab
- 3D printed case - Free from Cornell RPL
- 16x32 LED Panel - Provided by Professor
- Push Buttons - $5.99
- Elegoo Mega 2560 - Already owned
- Resistors and Wires - Provided in Lab
Total: $69.95
References
[1] DrumBit App[2] PyGame Library
[3] Live Scoreboard on the Raspberry Pi
[4] Button Matrix
[5] LED Panel Datasheet
[6] PyDub Library
[7] Scipy Library
[8] PyAudio
[9] Pi Music Synthesizer
[10] Pigpio Library
Code Appendix
# This function finds the button that was pressed. When the Pi receives a signal form the serial port
# connected to the microcontroller, it reads the string. The output from the microcontroller is the
# row and column of the button that was pressed. The function reads that and then toggles the current
# state of the button. If it is pressed and the sound was off, the button will not indicate true and color
# the button blue, indicating the button was pressed. If the button is then pressed again, it will return the
# state to false, and turn off the button.This will color it pink. All buttons are initially set to false
def uno_to_pi(but_matrix, serial, on_color, off_color, graphics,matrix):
# Decoding the string from serial
line = serial.readline().decode().strip()
row, col = map(int, line.split(','))
# Changing the state of the button
but_matrix[row - 1][col - 1] = not but_matrix[row - 1][col - 1]
# Determining LEDs associated with the button and changing the color
x_top_left_box = 30 - (col - 1) * 2
y_top_left_box = 14 - (row - 1)* 2
x_bottom_right_box = x_top_left_box + 1
y_bottom_right_box = y_top_left_box + 1
if but_matrix[row-1][col-1] == True:
graphics.DrawLine(matrix,x_top_left_box, y_top_left_box, x_bottom_right_box, y_top_left_box, on_color)
graphics.DrawLine(matrix,x_top_left_box, y_bottom_right_box, x_bottom_right_box, y_bottom_right_box, on_color)
else:
graphics.DrawLine(matrix,x_top_left_box, y_top_left_box, x_bottom_right_box, y_top_left_box, off_color)
graphics.DrawLine(matrix,x_top_left_box, y_bottom_right_box, x_bottom_right_box, y_bottom_right_box, off_color)
# Libraries necessary for program to run
import pygame, pigame # Import pygame graphics library
from pygame.locals import *
import os # for OS calls
import RPi.GPIO as GPIO
import time
import sys
import threading
# For LED panel
from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics
from PIL import Image
# For button matrix -- initializes a serial port to read theoutput from the microcontroller that
# handles the button matrix
import serial
global ser
ser = serial.Serial('/dev/ttyACM0', 115200)
# Function that fins the button that is pressed
import uno_to_pi
## Schedule a higher priority for python
param = os.sched_param(os.sched_get_priority_max(os.SCHED_FIFO))
os.sched_setscheduler(0,os.SCHED_FIFO,param)
# Initializes the GPIO buttons - used in previous iterations
GPIO.setmode(GPIO.BCM)
#piTFT display stuff - uncomment to put pygame on piTFT
# os.putenv('SDL_VIDEODRV','fbcon') # Display on piTFT
# os.putenv('SDL_FBDEV','/dev/fb1')
# os.putenv('SDL_MOUSEDRV','dummy')
# os.putenv('SDL_MOUSEDEV','/dev/null')
# os.putenv('DISPLAY','')
# Initialize pygame, pygame.mixer; pygame.mixed.init is used to change the buffer size
# to 512, which minimizes delay in the audio
pygame.mixer.pre_init(22050, -16, 2, 512)
pygame.init()
pygame.mixer.quit
pygame.mixer.init(22050,-16,2,512)
# pygame.mouse.set_visible(False) #uncomment for ipTFT
# piTFT initializations
size = width, height = 320, 240
pitft = pigame.PiTft()
lcd = pygame.display.set_mode(size)
lcd.fill((0,0,0))
pygame.display.update()
font_big = pygame.font.Font(None,40)
font_L2 = pygame.font.Font(None, 30)
# Colors used in the program. First 2 are for the piTFT and the last three are
# for the LED panel
WHITE = (255, 255, 255)
RED = (255, 0, 0)
blue = graphics.Color(0,0, 255)
pink = graphics.Color(255,20,147)
whiteled = graphics.Color(255,255,255)
###### Instruments - Load Audio ###########
# Audio Information:
# The LED panel allows us to have 8 rows, which in turn means that we have 8 tracks. The track is an editor for
# an individual audio. Each track is one row of the led panel. Each track has 16 beats, associated with a button, that
# when pressed, will toggle the audio to play at that specific beat.
#
# Currently, there are 13 instruments. Each instrument has an audio file associated with it. To add more instruments,
# add more wav files to audio_files
#
# The idea here is that the tracks and instruments are independent of each other. The track is the base of our
# program. There are always the same 8 tracks. However, we are able to change the instrument associated with each track.
#List of instruments and the audio file associated with that instrument
instruments = ['Bass Drum', 'Crash Cymbal', 'Cymbal', 'Hi Hat with stick', 'Hi Hat 1',
'Hi Hat 2', 'Side Snare 1', 'Side Snare 2', 'Snare', 'Snare Tom',
'Tom 1', 'Tom 2', 'Tom 3']
instr_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
instr_list_index = 0
# Older audio imported files -- not used anymore
# OLDER_audio_files = ['Bass_drum.wav','Crash_cymbal.wav','Cymbal.wav','Hi_hat_with_stick.wav',
# 'Hit_hat_1.wav','Hit_hat_2.wav','Side_snare_1.wav','Side_snare_2.wav',
# 'Snare.wav','Snare_Tom.wav','Tom_1.wav','Tom_2.wav','Tom_3.wav']
# OLD_audio_files = ['Bass_drum_V2.wav','Crash_cymbal_V2.wav','Cymbal_V2.wav','Hi_hat_with_stick_V2.wav',
# 'Hit_hat_1_V2.wav','Hit_hat_2_V2.wav','Side_snare_1_V2.wav','Side_snare_2_V2.wav',
# 'Snare_V2.wav','Snare_Tom_V2.wav','Tom_1_V2.wav','Tom_2_V2.wav','Tom_3_V2.wav']
audio_files = ['Bass_drum_V3.wav','Crash_cymbal_V3.wav','Cymbal_V3.wav','Hi_hat_with_stick_V3.wav',
'Hi_hat_1_V3.wav','Hi_hat_2_V3.wav','Snare_V3.wav','Side_snare_1_V3.wav','Side_snare_2_V3.wav',
'Snare_Tom_V3.wav','Tom_1_V3.wav','Tom_2_V3.wav','Tom_3_V3.wav']
#List of tracks
track_list = [1, 2, 3, 4, 5, 6, 7, 8]
track_list_index = 0
# all_chennels is the array storing the audio files ready to play
all_channels = []
# This loop stores the audio files in an playable way for pygame. The index of all_channels is the same
# as the index for the instruments list.
for audio_int in range(len(audio_files)):
# print(audio_files[audio_int])
channel = pygame.mixer.Sound(audio_files[audio_int])
all_channels.append(channel)
# Initialize the current channels associated with each track. The index of curr_channels_list is the same
# as the index of the tracks list
curr_channels_list = []
# Initialize the tracks to have the first 8 instruments in the instrument list.
for i in range(8):
curr_channels_list.append(all_channels[i])
#beats per minute time calc
bpm = 120
# bpm_display = bpm
bpm_time_sec = 65/bpm
button_matrix = [[False] *16 for i in range(8)] # each button is initialized as false, so sound is not played
print(len(button_matrix))
print(len(button_matrix[1]))
########### Initialize piTFT Display ###########
# touch buttons
start_touch_buttons = {'Start': (40, 220), 'Quit': (280, 220)}
level_two_buttons = {'Track #' + str(track_list[track_list_index]):(160, 140), 'Up': (160, 110),
'Down': (160, 170), 'Back':(260,200),
'GO': (60, 110), 'BPM': (60, 170), '<-': (30, 190), '->': (90, 190), str(bpm): (60, 190),
'Main Editor': (160, 40), 'Play':(260,80)} #'Play':(260,80), 'Pause': (260, 140),
level_three_buttons = {'Audio: ' + instruments[instr_list_index]:(160, 40),
str(track_list[track_list_index]): (160, 120), 'Switch': (160, 150), 'Save': (260,200)}
# volume button - removed from piTFT, use speaker volume toggle instead
# volume_bar = pygame.draw.rect(lcd, RED, (160, 120, 15, 100), 0)
lcd.fill((0,0,0))
for k,v in start_touch_buttons.items():
text_surface = font_big.render('%s'%k, True, WHITE)
rect = text_surface.get_rect(center=v)
lcd.blit(text_surface,rect)
logo = pygame.image.load("RasPiBeats_logo.png")
logo = pygame.transform.scale(logo, (150,160))
logo_rect = logo.get_rect()
logo_rect.center=(160,120)
lcd.blit(logo, logo_rect)
pygame.display.update()
pitft.update()
level_one = False
global level_two
level_two = True
level_three = False
########### Initialize LED Panel ###########
options = RGBMatrixOptions()
options.rows = 16
options.cols = 32
options.brightness = 50
options.gpio_slowdown = 0
options.chain_length = 1
options.parallel = 1
options.row_address_type = 0
options.multiplexing = 0
options.pwm_bits = 11
options.pwm_lsb_nanoseconds = 130
options.led_rgb_sequence = "RGB"
options.pixel_mapper_config = ""
options.panel_type = ""
options.gpio_slowdown = 4
options.drop_privileges = True
matrix = RGBMatrix(options = options)
matrix.CreateFrameCanvas()
matrix.Fill(255,20,147)
curr_column = 0 #1 - 16
last_column = 15
green_led = graphics.Color(0,255,0)
for i in range(len(button_matrix)):
for j in range(len(button_matrix[1])):
if (i == 3):
button_matrix[i][j] = True
if (i == 0):
if (j == 0 or j == 4 or j == 8 or j == 12):
button_matrix[i][j] = True
if (i == 6):
if (j == 2 or j == 6 or j == 10 or j == 14):
button_matrix[i][j] = True
if (i == 1 or i == 2):
if (j == 15):
button_matrix[i][j] = True
x_top_left_box = 30 - j * 2
y_top_left_box = 14 - i* 2
x_bottom_right_box = x_top_left_box + 1
y_bottom_right_box = y_top_left_box + 1
if button_matrix[i][j] == True:
graphics.DrawLine(matrix,x_top_left_box, y_top_left_box, x_bottom_right_box, y_top_left_box, blue)
graphics.DrawLine(matrix,x_top_left_box, y_bottom_right_box, x_bottom_right_box, y_bottom_right_box, blue)
else:
graphics.DrawLine(matrix,x_top_left_box, y_top_left_box, x_bottom_right_box, y_top_left_box, pink)
graphics.DrawLine(matrix,x_top_left_box, y_bottom_right_box, x_bottom_right_box, y_bottom_right_box, pink)
# Flags that toggle whether the code is run and the music is played
Beats_Run = False
play_flag = False
Setup_Run = True
######## Event Callback section for better timing ###########
# This callback function runs the LED panel and the audio playback. The function runs through
# each row in the current column (the column in this case is the current beat) and determines
# if the button was pressed. If it has been pressed, then the LED's associated with the button will light
# up to a different color than the base color, and the audio will play.
def thread_function():
global running
running = True
def timer_callback():
while running:
# print(bpm)
if Beats_Run:
# uses the global current column, color, and the current channel list
global curr_column, last_column, whiteled, green_led, curr_channels_list, bpm_time_sec
time.sleep(bpm_time_sec)
# Stops playing the current audio
# for channel in curr_channels_list:
# channel.stop()
# Returns the led's to the original color
# matrix.Fill(255,0,0)
# Sanity check to make sure curr_column is the correct range - probably don't need. I don't
# remember why it is here.
# if (curr_column <= 16): # and (level_two == True):
# Runs through each track and determines if the button is pressed
# print("Matrix on column: ", curr_column)
for channel_index in range(8):
# print(channel_index)
# print(last_column -1)
# print(button_matrix[channel_index][last_column])
if (button_matrix[channel_index][last_column] == False):
prev_color = pink
x_top_left_box = 30 - last_column * 2
y_top_left_box = 14 - channel_index * 2
x_bottom_right_box = x_top_left_box + 1
y_bottom_right_box = y_top_left_box + 1
graphics.DrawLine(matrix,x_top_left_box, y_top_left_box, x_bottom_right_box, y_top_left_box, prev_color)
graphics.DrawLine(matrix,x_top_left_box, y_bottom_right_box, x_bottom_right_box, y_bottom_right_box, prev_color)
# button_pressed will be determined by the arduino. A button press will
# toggle a change in the array of the buttons. This code needs to be written.
# This code should read a serial monitor from the arduino that outputs the button number
else:
channel = curr_channels_list[channel_index]
channel.stop()
if (button_matrix[channel_index][curr_column]):
# These two lines play the audio associated with the button.
# The code should run quick enough that the tracks will sound
# like they play at the same time
channel = curr_channels_list[channel_index]
#print(channel.get_length())
channel.play()
else:
x_top_left_box = 30 - curr_column * 2
y_top_left_box = 14 - channel_index * 2
x_bottom_right_box = x_top_left_box + 1
y_bottom_right_box = y_top_left_box + 1
graphics.DrawLine(matrix,x_top_left_box, y_top_left_box, x_bottom_right_box, y_top_left_box, whiteled)
graphics.DrawLine(matrix,x_top_left_box, y_bottom_right_box, x_bottom_right_box, y_bottom_right_box, whiteled)
# changes the column to the next one
if curr_column == 15:
curr_column = 0
else:
curr_column +=1
if last_column == 15:
last_column = 0
else:
last_column +=1
thread = threading.Thread(target=thread_function)
thread.start()
thread_timer = threading.Thread(target=timer_callback)
thread_timer.start()
currTime = time.time()
t_end = currTime + 10000
while (Setup_Run and currTime < t_end):
#################### piTFT display stuff ##########################
pygame.display.update()
pitft.update()
if (ser.in_waiting > 0): # If there is something in the serial monitor, run the function
uno_to_pi.uno_to_pi(button_matrix, ser, blue, pink, graphics,matrix)
if (level_one):
# Screen display for level one - Opening screen
lcd.fill((0,0,0))
for k,v in start_touch_buttons.items():
text_surface = font_big.render('%s'%k, True, WHITE)
rect = text_surface.get_rect(center=v)
lcd.blit(text_surface,rect)
logo = pygame.image.load("RasPiBeats_logo.png")
logo = pygame.transform.scale(logo, (150,160))
logo_rect = logo.get_rect()
logo_rect.center=(160,120)
lcd.blit(logo, logo_rect)
for event in pygame.event.get():
if (event.type is MOUSEBUTTONDOWN):
x,y = pygame.mouse.get_pos()
elif(event.type is MOUSEBUTTONUP):
x,y = pygame.mouse.get_pos()
if y > 200 and (x > 0 and x < 80): # Start - move to the next screen
print("Level Two")
level_two = True
level_one = False
elif y > 200 and (x > 240 and x < 320): # Quit
print("QUIT")
Beats_Run = False
elif (level_two):
# Screen display for level two - Main screen
lcd.fill((0,0,0))
for k,v in level_two_buttons.items():
text_surface = font_L2.render('%s'%k, True, WHITE)
rect = text_surface.get_rect(center=v)
lcd.blit(text_surface,rect)
for event in pygame.event.get():
if (event.type is MOUSEBUTTONDOWN):
x,y = pygame.mouse.get_pos()
#print("event on level 2", x,y)
elif(event.type is MOUSEBUTTONUP):
x,y = pygame.mouse.get_pos()
print(x,y)
if (x > 225 and x < 300) and (y > 180 and y < 215): #Back
level_one = True
level_two = False
#print("Audio saved!")
elif (x > 120 and x < 200) and (y > 150 and y < 180): # Up
if (track_list_index != 0):
track_list_index -= 1
elif (x > 125 and x < 200) and (y > 90 and y < 125): # Down
if (track_list_index != len(track_list)):
track_list_index += 1
elif (x > 10 and x < 110) and (y > 80 and y < 140): #GO
level_three = True
level_two = False
elif (x > 80 and x < 110) and (y > 170 and y < 240): #increase BPM
bpm += 10
bpm_time_sec = 60/bpm
print(bpm_time_sec)
elif (x > 7 and x < 50 ) and (y > 170 and y < 240): #decrease BPM
bpm -= 10
bpm_time_sec = 60/bpm
print(bpm_time_sec)
elif (x > 220 and x < 280) and (y > 60 and y < 100): # 'Play':(260,80)
Setup_Run = False
Beats_Run = True
del(pitft)
pygame.display.quit()
level_two_buttons = {'Track #' + str(track_list[track_list_index]):(160, 140), 'Up': (160, 110),
'Down': (160, 170), 'Back':(260,200),
'GO': (60, 110), 'BPM': (60, 170), '<-': (30, 190), '->': (90, 190), str(bpm): (60, 190),
'Main Editor': (160, 40), 'Play':(260,80)} #'Play':(260,80), 'Pause': (260, 140),
elif (level_three):
# Screen display for level three - track screen
lcd.fill((0,0,0))
for k,v in level_three_buttons.items():
text_surface = font_L2.render('%s'%k, True, WHITE)
rect = text_surface.get_rect(center=v)
lcd.blit(text_surface,rect)
for event in pygame.event.get():
if (event.type is MOUSEBUTTONDOWN):
x,y = pygame.mouse.get_pos()
print("event on level 2", x,y)
elif(event.type is MOUSEBUTTONUP):
x,y = pygame.mouse.get_pos()
print(x,y)
if (x > 100 and x < 220) and (y > 120 and y < 180): # Switch Sound
if (instr_list_index + 1 == len(instr_list)):
instr_list_index = 0
else:
instr_list_index += 1
all_channels[instr_list_index].play()
elif (x > 200 and x < 320) and (y > 180 and y < 240): #Save
print("Audio saved!")
level_two = True
level_three = False
curr_channels_list[track_list_index] = all_channels[instr_list_index]
level_three_buttons = {'Audio: ' + instruments[instr_list_index]:(160, 40),
str(track_list[track_list_index]): (160, 120), 'Switch': (160, 150), 'Save': (260,200)}
# pygame.display.update()
# pitft.update()
currTime = time.time()
#print("time = ", t_end - currTime)
######################## End of piTFT display sutff ###############################
###################################################################################
###################################################################################
while (Beats_Run and currTime < t_end):
currTime = time.time()
if (currTime+1 > t_end):
Beats_Run = False
print("timeout")
running = False
thread.join()
thread_timer.join()
lcd.fill((0,0,0))
pygame.display.update()
pitft.update()
print("quit")
# del(pitft)
# pygame.display.quit()
pygame.quit()
GPIO.cleanup()
print("finished code")
# Libraries necessary for program to run
import pygame, pigame # Import pygame graphics library
from pygame.locals import *
import os # for OS calls
import RPi.GPIO as GPIO
import time
import sys
import threading
# For LED panel
from rgbmatrix import RGBMatrix, RGBMatrixOptions, graphics
from PIL import Image
# For button matrix -- initializes a serial port to read theoutput from the microcontroller that
# handles the button matrix
import serial
global ser
ser = serial.Serial('/dev/ttyACM0', 115200)
# Function that fins the button that is pressed
import uno_to_pi
## Schedule a higher priority for python
param = os.sched_param(os.sched_get_priority_max(os.SCHED_FIFO))
os.sched_setscheduler(0,os.SCHED_FIFO,param)
# Initializes the GPIO buttons - used in previous iterations
GPIO.setmode(GPIO.BCM)
#piTFT display stuff - uncomment to put pygame on piTFT
os.putenv('SDL_VIDEODRV','fbcon') # Display on piTFT
os.putenv('SDL_FBDEV','/dev/fb1')
os.putenv('SDL_MOUSEDRV','dummy')
os.putenv('SDL_MOUSEDEV','/dev/null')
os.putenv('DISPLAY','')
# Initialize pygame, pygame.mixer; pygame.mixed.init is used to change the buffer size
# to 512, which minimizes delay in the audio
pygame.mixer.pre_init(22050, -16, 2, 512)
pygame.init()
pygame.mixer.quit
pygame.mixer.init(22050,-16,2,512)
pygame.mouse.set_visible(False) #uncomment for ipTFT
# piTFT initializations
size = width, height = 320, 240
pitft = pigame.PiTft()
lcd = pygame.display.set_mode(size)
lcd.fill((0,0,0))
pygame.display.update()
font_big = pygame.font.Font(None,40)
font_L2 = pygame.font.Font(None, 30)
# Colors used in the program. First 2 are for the piTFT and the last three are
# for the LED panel
WHITE = (255, 255, 255)
RED = (255, 0, 0)
blue = graphics.Color(0,0, 255)
pink = graphics.Color(255,20,147)
whiteled = graphics.Color(255,255,255)
###### Instruments - Load Audio ###########
# Audio Information:
# The LED panel allows us to have 8 rows, which in turn means that we have 8 tracks. The track is an editor for
# an individual audio. Each track is one row of the led panel. Each track has 16 beats, associated with a button, that
# when pressed, will toggle the audio to play at that specific beat.
#
# Currently, there are 13 instruments. Each instrument has an audio file associated with it. To add more instruments,
# add more wav files to audio_files
#
# The idea here is that the tracks and instruments are independent of each other. The track is the base of our
# program. There are always the same 8 tracks. However, we are able to change the instrument associated with each track.
#List of instruments and the audio file associated with that instrument
instruments = ['Bass Drum', 'Crash Cymbal', 'Cymbal', 'Hi Hat with stick', 'Hi Hat 1',
'Hi Hat 2', 'Side Snare 1', 'Side Snare 2', 'Snare', 'Snare Tom',
'Tom 1', 'Tom 2', 'Tom 3']
instr_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
instr_list_index = 0
# Older audio imported files -- not used anymore
# OLDER_audio_files = ['Bass_drum.wav','Crash_cymbal.wav','Cymbal.wav','Hi_hat_with_stick.wav',
# 'Hit_hat_1.wav','Hit_hat_2.wav','Side_snare_1.wav','Side_snare_2.wav',
# 'Snare.wav','Snare_Tom.wav','Tom_1.wav','Tom_2.wav','Tom_3.wav']
# OLD_audio_files = ['Bass_drum_V2.wav','Crash_cymbal_V2.wav','Cymbal_V2.wav','Hi_hat_with_stick_V2.wav',
# 'Hit_hat_1_V2.wav','Hit_hat_2_V2.wav','Side_snare_1_V2.wav','Side_snare_2_V2.wav',
# 'Snare_V2.wav','Snare_Tom_V2.wav','Tom_1_V2.wav','Tom_2_V2.wav','Tom_3_V2.wav']
audio_files = ['Bass_drum_V3.wav','Crash_cymbal_V3.wav','Cymbal_V3.wav','Hi_hat_with_stick_V3.wav',
'Hi_hat_1_V3.wav','Hi_hat_2_V3.wav','Side_snare_1_V3.wav','Side_snare_2_V3.wav',
'Snare_V3.wav','Snare_Tom_V3.wav','Tom_1_V3.wav','Tom_2_V3.wav','Tom_3_V3.wav']
#List of tracks
track_list = [1, 2, 3, 4, 5, 6, 7, 8]
track_list_index = 0
# all_chennels is the array storing the audio files ready to play
all_channels = []
# This loop stores the audio files in an playable way for pygame. The index of all_channels is the same
# as the index for the instruments list.
for audio_int in range(len(audio_files)):
# print(audio_files[audio_int])
channel = pygame.mixer.Sound(audio_files[audio_int])
all_channels.append(channel)
# Initialize the current channels associated with each track. The index of curr_channels_list is the same
# as the index of the tracks list
curr_channels_list = []
# Initialize the tracks to have the first 8 instruments in the instrument list.
for i in range(8):
curr_channels_list.append(all_channels[i])
#beats per minute time calc
bpm = 60
# bpm_display = bpm
bpm_time_sec = 65/bpm
button_matrix = [[False] *16 for i in range(8)] # each button is initialized as false, so sound is not played
########### Initialize piTFT Display ###########
# touch buttons
start_touch_buttons = {'Start': (40, 220), 'Quit': (280, 220)}
level_two_buttons = {'Track #' + str(track_list[track_list_index]):(160, 140), 'Up': (160, 110),
'Down': (160, 170), 'Back':(260,200),
'GO': (60, 110), 'BPM': (60, 170), '<-': (30, 190), '->': (90, 190), str(bpm): (60, 190),
'Main Editor': (160, 40)} #'Play':(260,80), 'Pause': (260, 140),
level_three_buttons = {'Audio: ' + instruments[instr_list_index]:(160, 40),
str(track_list[track_list_index]): (160, 120), 'Switch': (160, 150), 'Save': (260,200)}
# volume button - removed from piTFT, use speaker volume toggle instead
# volume_bar = pygame.draw.rect(lcd, RED, (160, 120, 15, 100), 0)
lcd.fill((0,0,0))
for k,v in start_touch_buttons.items():
text_surface = font_big.render('%s'%k, True, WHITE)
rect = text_surface.get_rect(center=v)
lcd.blit(text_surface,rect)
logo = pygame.image.load("RasPiBeats_logo.png")
logo = pygame.transform.scale(logo, (150,160))
logo_rect = logo.get_rect()
logo_rect.center=(160,120)
lcd.blit(logo, logo_rect)
pygame.display.update()
pitft.update()
level_one = True
global level_two
level_two = False
level_three = False
########### Initialize LED Panel ###########
options = RGBMatrixOptions()
options.rows = 16
options.cols = 32
options.brightness = 50
options.gpio_slowdown = 0
options.chain_length = 1
options.parallel = 1
options.row_address_type = 0
options.multiplexing = 0
options.pwm_bits = 11
options.pwm_lsb_nanoseconds = 130
options.led_rgb_sequence = "RGB"
options.pixel_mapper_config = ""
options.panel_type = ""
options.gpio_slowdown = 4
options.drop_privileges = True
matrix = RGBMatrix(options = options)
matrix.CreateFrameCanvas()
matrix.Fill(255,20,147)
curr_column = 0 #1 - 16
last_column = 15
green_led = graphics.Color(0,255,0)
######## Event Callback section for better timing ###########
# This callback function runs the LED panel and the audio playback. The function runs through
# each row in the current column (the column in this case is the current beat) and determines
# if the button was pressed. If it has been pressed, then the LED's associated with the button will light
# up to a different color than the base color, and the audio will play.
def thread_function():
global running
running = True
def timer_callback():
while running:
# print(bpm)
if level_two:
# uses the global current column, color, and the current channel list
global curr_column, last_column, whiteled, green_led, curr_channels_list, bpm_time_sec
time.sleep(bpm_time_sec)
# Stops playing the current audio
# for channel in curr_channels_list:
# channel.stop()
# Returns the led's to the original color
# matrix.Fill(255,0,0)
# Sanity check to make sure curr_column is the correct range - probably don't need. I don't
# remember why it is here.
# if (curr_column <= 16): # and (level_two == True):
# Runs through each track and determines if the button is pressed
# print("Matrix on column: ", curr_column)
for channel_index in range(8):
# print(channel_index)
# print(last_column -1)
# print(button_matrix[channel_index][last_column])
if (button_matrix[channel_index][last_column] == False):
prev_color = pink
x_top_left_box = 30 - last_column * 2
y_top_left_box = 14 - channel_index * 2
x_bottom_right_box = x_top_left_box + 1
y_bottom_right_box = y_top_left_box + 1
graphics.DrawLine(matrix,x_top_left_box, y_top_left_box, x_bottom_right_box, y_top_left_box, prev_color)
graphics.DrawLine(matrix,x_top_left_box, y_bottom_right_box, x_bottom_right_box, y_bottom_right_box, prev_color)
# button_pressed will be determined by the arduino. A button press will
# toggle a change in the array of the buttons. This code needs to be written.
# This code should read a serial monitor from the arduino that outputs the button number
else:
channel = curr_channels_list[channel_index]
channel.stop()
if (button_matrix[channel_index][curr_column]):
# These two lines play the audio associated with the button.
# The code should run quick enough that the tracks will sound
# like they play at the same time
channel = curr_channels_list[channel_index]
#print(channel.get_length())
channel.play()
else:
x_top_left_box = 30 - curr_column * 2
y_top_left_box = 14 - channel_index * 2
x_bottom_right_box = x_top_left_box + 1
y_bottom_right_box = y_top_left_box + 1
graphics.DrawLine(matrix,x_top_left_box, y_top_left_box, x_bottom_right_box, y_top_left_box, whiteled)
graphics.DrawLine(matrix,x_top_left_box, y_bottom_right_box, x_bottom_right_box, y_bottom_right_box, whiteled)
# changes the column to the next one
if curr_column == 15:
curr_column = 0
else:
curr_column +=1
if last_column == 15:
last_column = 0
else:
last_column +=1
thread = threading.Thread(target=thread_function)
thread.start()
thread_timer = threading.Thread(target=timer_callback)
thread_timer.start()
# Flags that toggle whether the code is run and the music is played
Beats_Run = True
play_flag = False
currTime = time.time()
t_end = currTime + 10000
while (Beats_Run and currTime < t_end):
#################### piTFT display stuff ##########################
pygame.display.update()
pitft.update()
if (ser.in_waiting > 0): # If there is something in the serial monitor, run the function
uno_to_pi.uno_to_pi(button_matrix, ser, blue, pink, graphics,matrix)
if (level_one):
# Screen display for level one - Opening screen
lcd.fill((0,0,0))
for k,v in start_touch_buttons.items():
text_surface = font_big.render('%s'%k, True, WHITE)
rect = text_surface.get_rect(center=v)
lcd.blit(text_surface,rect)
logo = pygame.image.load("RasPiBeats_logo.png")
logo = pygame.transform.scale(logo, (150,160))
logo_rect = logo.get_rect()
logo_rect.center=(160,120)
lcd.blit(logo, logo_rect)
for event in pygame.event.get():
if (event.type is MOUSEBUTTONDOWN):
x,y = pygame.mouse.get_pos()
elif(event.type is MOUSEBUTTONUP):
x,y = pygame.mouse.get_pos()
if y > 200 and (x > 0 and x < 80): # Start - move to the next screen
print("Level Two")
level_two = True
level_one = False
elif y > 200 and (x > 240 and x < 320): # Quit
print("QUIT")
Beats_Run = False
elif (level_two):
# Screen display for level two - Main screen
lcd.fill((0,0,0))
for k,v in level_two_buttons.items():
text_surface = font_L2.render('%s'%k, True, WHITE)
rect = text_surface.get_rect(center=v)
lcd.blit(text_surface,rect)
for event in pygame.event.get():
if (event.type is MOUSEBUTTONDOWN):
x,y = pygame.mouse.get_pos()
#print("event on level 2", x,y)
elif(event.type is MOUSEBUTTONUP):
x,y = pygame.mouse.get_pos()
print(x,y)
if (x > 225 and x < 300) and (y > 180 and y < 215): #Back
level_one = True
level_two = False
#print("Audio saved!")
elif (x > 120 and x < 200) and (y > 150 and y < 180): # Up
if (track_list_index != 0):
track_list_index -= 1
elif (x > 125 and x < 200) and (y > 90 and y < 125): # Down
if (track_list_index != len(track_list)):
track_list_index += 1
elif (x > 10 and x < 110) and (y > 80 and y < 140): #GO
level_three = True
level_two = False
elif (x > 80 and x < 110) and (y > 170 and y < 240): #increase BPM
bpm += 10
bpm_time_sec = 60/bpm
print(bpm_time_sec)
elif (x > 7 and x < 50 ) and (y > 170 and y < 240): #decrease BPM
bpm -= 10
bpm_time_sec = 60/bpm
print(bpm_time_sec)
level_two_buttons = {'Track #' + str(track_list[track_list_index]):(160, 140), 'Up': (160, 110),
'Down': (160, 170), 'Back':(260,200),
'GO': (60, 110), 'BPM': (60, 170), '<-': (30, 190), '->': (90, 190), str(bpm): (60, 190),
'Main Editor': (160, 40)} #'Play':(260,80), 'Pause': (260, 140),
elif (level_three):
# Screen display for level three - track screen
lcd.fill((0,0,0))
for k,v in level_three_buttons.items():
text_surface = font_L2.render('%s'%k, True, WHITE)
rect = text_surface.get_rect(center=v)
lcd.blit(text_surface,rect)
for event in pygame.event.get():
if (event.type is MOUSEBUTTONDOWN):
x,y = pygame.mouse.get_pos()
print("event on level 2", x,y)
elif(event.type is MOUSEBUTTONUP):
x,y = pygame.mouse.get_pos()
print(x,y)
if (x > 100 and x < 220) and (y > 120 and y < 180): # Switch Sound
if (instr_list_index + 1 == len(instr_list)):
instr_list_index = 0
else:
instr_list_index += 1
all_channels[instr_list_index].play()
elif (x > 200 and x < 320) and (y > 180 and y < 240): #Save
print("Audio saved!")
level_two = True
level_three = False
curr_channels_list[track_list_index] = all_channels[instr_list_index]
level_three_buttons = {'Audio: ' + instruments[instr_list_index]:(160, 40),
str(track_list[track_list_index]): (160, 120), 'Switch': (160, 150), 'Save': (260,200)}
pygame.display.update()
pitft.update()
currTime = time.time()
#print("time = ", t_end - currTime)
if (currTime+1 > t_end):
Beats_Run = False
print("timeout")
######################## End of piTFT display sutff ###############################
###################################################################################
###################################################################################
running = False
thread.join()
thread_timer.join()
lcd.fill((0,0,0))
pygame.display.update()
pitft.update()
print("quit")
del(pitft)
pygame.display.quit()
pygame.quit()
GPIO.cleanup()
print("finished code")